feat(examples): add Kubernetes GPU autoscaling recipe - #111

Thanks for moving this Kubernetes GPU autoscaling work into the community repository. The static checks and contract tests are in good shape, but two items need to be completed before approval:

1. Please provide a documented live Kubernetes/GPU validation of the core workflow. The recipe's primary behavior is GPU-backed inference and HPA scaling, but the PR currently states that no live Kubernetes/H100 validation was run. At minimum, validate deployment, authenticated inference, the custom GPU metric, and a scale-up/scale-down cycle on a suitable cluster.
2. Please complete the contributor review of the agent-generated text and check the corresponding PR-template item. The PR currently marks this as required before merge.

Once those are complete, we can reassess the PR for approval.

Thank you for the update. The documented 4× L40S run now covers the chart deployment, authenticated inference, GPU metric, HPA scale-up/scale-down, and Envoy LeastRequest distribution. The contributor and documentation-review checklist items are also complete on the current head.

Two items still need attention before approval:

1. Separate the internal cleartext route from externally exposed Envoy Services. The current Gateway always creates a hostname-unrestricted HTTP listener and OpenShell route on port 80. When ingress.gateway.serviceType is changed to NodePort or LoadBalancer, that same listener becomes externally reachable even when TLS is configured. Requests using another Host value or the load-balancer IP can select the OpenShell route instead of the hostname-specific HTTPS redirect, bypassing Gateway TLS and Basic authentication while carrying the reusable Bearer credential and inference traffic over cleartext. Please either use a separate ClusterIP-only Gateway/dataplane for OpenShell and a distinct TLS-only external Gateway, or reject NodePort/LoadBalancer while the internal HTTP listener is present. Add a render/security contract test for the chosen boundary.
2. Complete or narrow the end-to-end validation claim. The L40S evidence validates the GPU chart and Envoy path, but the PR still advertises a NemoClaw/OpenShell sandbox workflow and explicitly states that sandbox creation was not live-validated. Please run the actual sandbox creation and inference path through https://inference.local, including a model request from the sandbox, or narrow the recipe's advertised scope so the unvalidated sandbox integration is not presented as a completed workflow.

The available local security, Kubernetes, inference-authentication, syntax, SPDX, taxonomy, and diff checks pass on 196fba6.

Thank you for the latest updates. The earlier Envoy exposure and end-to-end OpenShell validation blockers are resolved, and the available DCO, SPDX, taxonomy, syntax, authentication, Kubernetes, and security checks pass on 0c25e5a.

Two items remain before approval:

1. Fix the broken upstream chart link. The README links NemoClaw GPU autoscaling chart to NVIDIA/NemoClaw/tree/main/deploy/helm/gpu_autoscaling_k8s, but that path returns 404 because NVIDIA/NemoClaw#7459 is still open. Please link to the immutable source commit (77334ccbbadba2c5079fe9e99fe80a3cddc846b5) or directly to PR #7459 instead of main.
2. Reject invalid Gateway path types. nemoclaw-gpu.httpPathMatchType currently maps every value other than Exact to PathPrefix. A typo or unsupported value therefore silently broadens the route instead of failing. Please accept only the supported Prefix and Exact inputs, fail rendering for anything else, and add a render-contract case for the invalid-value failure.

Once these are addressed and the final documentation-review evidence identifies the exact reviewed head, this should be ready for approval.

Thank you for the additional validation and documentation updates. The 4× L40S result is sufficient for this experimental recipe; H100 validation can remain follow-up work and does not need to block this PR. The available repository checks pass on b4d8dc6.

The previously requested items are still outstanding:

1. The README’s NemoClaw GPU autoscaling chart link still points to main, where that path returns 404. Please use the immutable 77334ccbbadba2c5079fe9e99fe80a3cddc846b5 source or NVIDIA/NemoClaw#7459.
2. nemoclaw-gpu.httpPathMatchType still maps every value other than Exact to PathPrefix. Please accept only Prefix and Exact, fail for unsupported values, and add the requested invalid-value render test.
3. The documentation-review evidence still says “see latest commits” instead of identifying the exact final reviewed head.

The latest commits also add switchable latency and request-rate HPA modes. This is a substantial, unvalidated scope expansion, so I recommend removing it from this PR and landing it separately after focused validation. In its current form:

• The rolling latency gauges retain their last samples indefinitely; a high value can remain after traffic stops and prevent scale-down.
• The nemoclaw_llm_request_rate adapter metricsQuery omits result="success", so it can aggregate error requests despite being documented as successful-completion rate.
• hpa-load-test.sh remains GPU-utilization-specific in its load model and reporting, so it does not validate the alternate modes it now advertises.
• Render coverage exercises only latency_p95, not every new mode, invalid mode handling, or the adapter-query semantics.

Please keep this PR on the validated GPU-utilization path, address the three outstanding procedural/render items, and move alternate HPA signals to a follow-up PR with targeted tests and live scale-up/scale-down evidence.

Thanks for the latest updates. The values-overlay consolidation is internally consistent, and the available repository, syntax, security, Kubernetes, and authentication contract checks pass on 5146e8b.

The requested changes are still outstanding:

1. The README link for the NemoClaw GPU autoscaling chart still points to a path on main that returns 404. Please link to immutable source commit 77334ccbbadba2c5079fe9e99fe80a3cddc846b5 or NVIDIA/NemoClaw#7459.
2. nemoclaw-gpu.httpPathMatchType still maps every value other than Exact to PathPrefix. Please accept only Prefix and Exact, fail rendering for anything else, and add the invalid-value render-contract test.
3. The documentation-review evidence still says “see latest commits” rather than naming the exact reviewed head.
4. The alternate latency and request-rate HPA modes remain in this PR without the requested focused validation. Please move them to a follow-up PR, or address the stale latency-gauge behavior, successful-request query filtering, mode-aware load testing, and complete render/adapter coverage.
5. The newly restored screenshots show a specific Brev host name and local workspace path in the terminal prompt. Please crop or redact those prompt lines before publishing them; this also needs to align with the PR declaration that nonpublic host names and workspace paths are excluded.

The branch is also one commit behind main; please update it after the content changes are complete.

Thanks for the updates. The branch is current with main, the Gateway path-type and exposure safeguards are fixed, the screenshots are sanitized, and the local security, authentication, Kubernetes, and Helm render contracts pass on 7ecfd92.

Three items remain before approval:

1. Fix the latency metric lifecycle. agent-metrics.ts retains the last rolling latency samples indefinitely. If the final requests remain above the configured target after load stops, the custom metric can remain high and prevent HPA scale-down. A successful run can depend on the last samples happening to fall below the target. Please add an idle-expiration/reset behavior with a focused test, or remove latency as a selectable HPA signal.
2. Align supported HPA modes and test coverage. values.yaml and the chart helpers still expose latency_p50, latency_p95, and request_rate, while the render contract tests only latency_avg. Please either restrict the built-in modes to the live-validated gpu and latency_avg signals, or add render/adapter contract coverage for every retained mode.
3. Correct the verification record:
   • git diff --check currently fails on trailing whitespace in README line 10.
   • The checked Helm command, helm lint . --set ingress.allowInsecureHttp=true, fails because ingress.auth.htpasswd is required. Please use a valid reproducible lint command or adjust the chart defaults.
   • Documentation review identifies 98c3bb4 as the reviewed head, but the current head is 7ecfd92. Please review the final changes and record the exact current head.

The prior source-path, Gateway path-type, screenshot-redaction, live-validation, and branch-update concerns are resolved.

Thanks for the thorough update. The three items from the previous review are resolved on 43b4f58: latency idle-expiration now has a focused passing contract, built-in HPA modes are restricted and tested, and the diff check, reproducible Helm lint, and documentation-review head are corrected. The repository, security, authentication, Kubernetes, metrics, and Helm render checks pass locally.

Two new blockers were introduced by this update:

1. Sanitize the committed screenshots again. docs/assets/hpa-scale-up.png and docs/assets/reference-4x-l40s.png show the specific Brev host name and a workspace path in the terminal prompt. This conflicts with the PR declaration that nonpublic host names and workspace paths are excluded. Please crop or redact those prompt lines. The latency screenshot still displays the legacy resource names but does not expose the host prompt.

2. Resolve and validate the Kubernetes resource rename. This commit renames the Deployment, Service, HPA, Gateway, Secrets, selectors, and scripts from *-agent to *-metrics-proxy. The supplied live evidence still shows nemoclaw-gpu-agent, so it does not validate the renamed topology. More importantly, the migration helper checks legacy labels on the new *-metrics-proxy Deployment name; an existing *-agent Deployment will not be found and removed, so an upgrade can leave both workloads running and competing for GPUs.

Please either:

• preserve the existing Kubernetes resource names and limit the clarity change to container labels/documentation, or
• implement a tested migration that detects and safely removes the old release resources, then provide live validation of install/upgrade, authenticated inference, HPA scale-up/down, Envoy distribution, and cleanup using the new names.

After the final screenshots or resource changes, update the documentation-review head again.

Thanks for the additional migration work and live evidence. The current-name migration behavior is now credible, and the branch remains current with main. Four items still need correction before approval:

1. Restore the sanitized screenshots. Commit 2f4db23 replaced the images with sanitized attachments, but 15442f9 reverted the README to docs/assets/reference-4x-l40s.png, docs/assets/hpa-scale-up.png, and the earlier latency attachment. The two committed images expose the specific Brev host name and workspace path, so the current README again conflicts with the release/compliance declaration.

2. Run migration before any Helm upgrade in every entry point. In hpa-load-test.sh, hpa_common_ensure_metrics_proxy_ready performs helm upgrade and waits for the new Deployment before hpa_common_migrate_pre_metrics_proxy_resources runs. hpa-reset.sh has the same ordering when DELETE_HPA=1. If the historical workload holds all available GPUs, the renamed workload can remain Pending and the script can fail before reaching migration. Move migration before hpa_common_ensure_metrics_proxy_ready in both scripts and add an ordering contract test.

3. Fix the security-contract cleanup. bash scripts/test-script-security-contract.sh currently prints its success line but exits nonzero because the EXIT trap leaves kubectl-migrate.log in TEST_TMP and rmdir fails. Include that file in cleanup or make cleanup remove the temporary directory safely, then verify the command returns exit code 0.

4. Update final review metadata. The PR records documentation review at 2f4db23, while the current head is 15442f9. After the final changes, review and record the exact final head.

The migration's legacy-resource detection and the contributor's live validation using *-metrics-proxy names are accepted; they do not need to be repeated unless the ordering fix changes runtime behavior.

Approved at 15193cfd351867549ca372d8fb3eadc5fe233d37.

All previously requested changes are resolved. I reviewed the final documentation-only updates, including the replica-count clarification, PromQL examples, and sanitized Grafana screenshots. The branch is current with main, the required checks pass, and there are no unresolved review threads.

The timestamp-style screenshot alt text can be improved later as a non-blocking documentation refinement.